Papers with tensor decomposition
Embedding Syntax and Semantics of Prepositions via Tensor Decomposition (N18-1)
Copied to clipboard
| Challenge: | Existing methods on preposition representation treat prepositions no different from content words (e.g., word2vec and GloVe). |
| Approach: | They propose to use word-triple counts to capture a preposition’s interaction with its attachment and complement and derive preposition embeddings via tensor decomposition on a large unlabeled corpus. |
| Outcome: | The proposed model is comparable to or better than the state-of-the-art on multiple standardized datasets. |
Mitigating Heterogeneity among Factor Tensors via Lie Group Manifolds for Tensor Decomposition Based Temporal Knowledge Graph Embedding (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have highlighted the effectiveness of tensor decomposition methods in the Temporal Knowledge Graphs Embedding task. |
| Approach: | They propose a method that maps factor tensors onto a unified smooth Lie group manifold to approximate homogeneous in tensian decomposition. |
| Outcome: | The proposed method can be directly integrated into existing tensor decomposition based TKGE methods without introducing extra parameters. |
PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work shows that probabilistic context-free grammars with neural parameterization can be effective in unsupervised constituency parsing. |
| Approach: | They propose a parameterization form of PCFGs based on tensor decomposition which has at most quadratic computational complexity in the symbol number. |
| Outcome: | The proposed model improves unsupervised constituency parsing performance across ten languages. |
Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to compress KV cache compromise precision or require extra data for calibration, limiting their practicality in LLM deployment. |
| Approach: | They propose a low-bit quantization technique based on tensor decomposition to effectively compress KV cache. |
| Outcome: | The proposed method reduces memory footprint and performance by 75% . it is compared with existing methods that compromise precision or require extra data for calibration . |
Low-Rank HOCA: Efficient High-Order Cross-Modal Attention for Video Captioning (D19-1)
Copied to clipboard
| Challenge: | Existing studies on video captioning focus on the association relationships between multiple modalities. |
| Approach: | They propose a video captioning model with high-order cross-modal attention (HOCA) they propose low-rank HOCA which adopts tensor decomposition to reduce the space requirement . |
| Outcome: | The proposed model captures cross-modal interaction of different modalities and reduces space requirement. |
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models (2022.coling-1)
Copied to clipboard
| Challenge: | Recent results show that the mix-of-experts architecture is parameter inefficient . large-scale pre-trained language models can achieve excellent performance in many NLP tasks. |
| Approach: | They propose to build a parameter-efficient mix-of-experts architecture by sharing information across experts. |
| Outcome: | The proposed architecture increases model capacity without increasing computation costs. |
Text Emotion Distribution Learning from Small Sample: A Meta-Learning Approach (D19-1)
Copied to clipboard
| Challenge: | Existing methods for text emotion distribution learning require a large amount of training data, which is difficult to obtain due to inconsistent perception of fine-grained emotion intensity. |
| Approach: | They propose a meta-learning approach to learn text emotion distributions from a small sample using tensor decomposition to capture contextual semantic similarity. |
| Outcome: | The proposed method outperforms state-of-the-art methods on a widely used EDL dataset. |
Uncertainty Quantification of Large Language Models through Multiple Uncertainty Sources (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for uncertainty quantification fail to capture multifaceted nature of natural language generation. |
| Approach: | They propose a multi-resource Uncertainty Quantification framework that integrates heterogeneous uncertainty signals into a unified measure. |
| Outcome: | The proposed framework outperforms existing methods on CoQA, NQ_Open, and HotpotQA. |
Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decomposition (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for MAS fail to address the unique complexities of multi-step reasoning . Existing uncertainty quantification methods struggle with cascading uncertainty . |
| Approach: | They propose a framework that quantifies uncertainty through tensor decomposition . they show that MATU effectively estimates holistic and robust uncertainty . |
| Outcome: | The proposed framework disentangles and quantifies distinct sources of uncertainty . it is generalizable across different agent structures and can be used for scientific discovery, education, healthcare and transportation. |